Weights can be thought of as the amount of influence the input has on the output. With weight initialization, we set the weights to random values to prevent the layer outputs or gradients from vanishing or exploding. To understand what vanishing and exploding gradients are, let's break down what a neural network does. In a neural network, many matrix multiplications are performed during forward propagation. In forward propagation, inputs are passed through layers to end up with a final output from the output node.

Recently, I've completed 30 Days of ML with Pytorch, where I explored all the major machine learning algorithms and basics of deep learning with concepts like activation, optimizer, loss functions etc. However, the idea behind starting 30 Days of ML with PyTorch is to learn the implementation of machine learning algorithm along with strengthening my grasp on PyTorch Library. So moving forward with this blog, I will share few important functions in PyTorch which remains part of every ML and DL algorithms. In neural networks, how we initialize our weights plays a great role in convergence of the model and a common but inefficient way to initialize weight is to declare weights as zeros, but other efficient way includes using uniform or normal or xaviers technique to initialize weights. It is applied for creating dropout functionality in neural network for regularization.

It's a mystery that deep learning works so well. Even though there are several hints about why deep neural networks are so effective, the truth is that nobody is entirely sure and theoretical understanding of deep learning is very much an active area of research. We will make neural networks paint abstract images for us, and then we will interpret those images to develop a better intuition on what might be happening under the hood. Also, as a bonus, by the end of the tutorial, you'll be able to generate images such as the following (everything is less than 100 lines of PyTorch code. This image was generated by a simple architecture called Compositional Pattern Producing Networks (CPPN) which I got introduced to via this blog post. In that blog post, the author generates abstract images via neural networks written in JavaScript.

It's a mystery that deep learning works so well. Even though there are several hints about why deep neural networks are so effective, the truth is that nobody is entirely sure and theoretical understanding of deep learning is very much an active area of research. We will make neural networks paint abstract images for us, and then we will interpret those images to develop a better intuition on what might be happening under the hood. Also, as a bonus, by the end of the tutorial, you'll be able to generate images such as the following (everything is less than 100 lines of PyTorch code. This image was generated by a simple architecture called Compositional Pattern Producing Networks (CPPN) which I got introduced to via this blog post. In that blog post, the author generates abstract images via neural networks written in JavaScript.